Project Aim

Begin to develop methods to collect accurate information about the facial expressions of elite tennis athletes during matchplay.

This beginning involves testing how well currently available facial recognition software performs at identifying faces of the two players in a tennis match.

Current Recognition Software

Current software has been intended for security purposes including:

• Access Points
• International Airports Passpoint points
• Watch List Screening in crowds

This presents issues as our use on Tennis Broadcasts provides multiple interchanging angles. This is unlike the intended use of a full frontal scan at an access point.

Challenges

• The multiple angles presents issues of whether the faces will be detectable
• There are many softwares available for purchase online. The most accessible during our time frame were APIs
• Softwares were intended for use beyond latent still detection, they were intended for real time detection and alerts. These softwares had capabilities far beyond what is needed

Opportunity

• We have a great opportunity to apply facial recognition software in an unusual and previously untested application
• Algorithms have certain capabilities that will allow for faces to be detected

Our Sample

105 Singles Matches from 2016 AO differing by:

• Gender, either Male or Female
• Court, one of 7 possible courts

We then took:

• 5 minute segments from the 105 Broadcast video files
• Of these segments, a still was taken at every 3 seconds

Computation Times for recognition of stills

• Skybiometry: 1.461
• Microsoft: 2.29
• Animetrics: 3.924

Examples of Application

The following slides include stills and what was recognised when the softwares were applied.

• Green: Microsoft
• Red: Animetrics
• Blue: Skybiometry

Accuracte Facial Recognition

The software that is chosen should be able to rcognise this face even though it is not a front on angle

Accuracte Facial Recognition

The software that is chosen should also be able to rcognise this face

Accuracte Facial Recognition

We would hope that this angle would still allow for recognition, however it was only recognised by one software,

Accuracte Facial Recognition

We would hope that this angle would still allow for recognition, however it was only recognised by one software

Accuracte Facial Recognition

We would hope that this size would still allow for recognition, however it was only recognised by one software, note that the minimum pixel distance that will allow for recognition is 36

Emotion Facial Recognition

As this is the intended future use we would need the software to detect emotions and angles such as this

Emotion Facial Recognition

This angle should also be detected by any software we would consider for this use

Emotion Facial Recognition

This angle should also be detected by any software we would consider for this use

Inaccuracte Facial Recognition

Inaccuracte Facial Recognition

Inaccuracte Facial Recognition

Interesting Recognition

Skybiometry was able to identify this ‘face’, created by the creases in the shirt

Interesting Recognition

Animetrics was able to identify two faces in this image, one being a towel

Interesting Recognition

Interesting Recognition

Crowd Recognition

Crowd Recognition

Crowd Recognition

Impractical Shots

Some shots result in difficulties when applying recognition software

Upward Angle

Birdseye View

Upward Angle

Birds-eye View

Obstructed Faces

Manual Recognition

Our Gold Standard to compare to the softwares selected

• Shiny App, using imager, shiny and shinyjs
• Scenes: live shot, person in the frame, situation, background, angle of the shot
• Faces: box, obscured, light on the face, angle of the head, glasses, or hat/visor

Still to come…

Comparisons of:

• Facial features and structures that influence recognition
• The resulting recogised areas

Questions and Recommendations